Discovering Predictive Association Rules

Association rule algorithms can produce a very large number of output patterns. This has raised questions of whether the set of discovered rules \over t" the data because all the patterns that satisfy some constraints are generated (the Bonferroni e ect). In other words, the question is whether some of the rules are \false discoveries" that are not statistically signi cant. We present a novel approach for estimating the number of \false discoveries" at any cuto level. Empirical evaluation shows that on typical datasets the fraction of rules that may be false discoveries is very small. A bonus of this work is that the statistical signi cance measures we compute are a good basis for ordering the rules for presentation to users, since they correspond to the statistical \surprise" of the rule. We also show how to compute con dence intervals for the support and con dence of an association rule, enabling the rule to be used predictively on future data.